Both three-dimensional image data in the form of an image data record, for example a computed tomography or magnetic resonance data record, and two-dimensional fluoroscopy images, e.g., x-ray images, of the target region of interest in the patient are available in many medical problems—be it in diagnostics or when carrying out minimally invasive procedures in particular. Here, it is desirable to be able to evaluate both the three-dimensional image data record and the two-dimensional x-ray image of the target region together, in particular, in the form of a fusion image or by integrating features of one image into the respective other image. To this end, it is necessary to relate the coordinate systems of the three-dimensional image data record and of the two-dimensional x-ray image to one another; this process may be referred to as registration, 2D/3D registration in the present case.
The 2D/3D registration is of particular importance in the case of image-guided medical procedures. Here, use is often made of x-ray apparatuses with a C-arm, on which an x-ray source and an x-ray detector are arranged opposite one another. Real-time monitoring of medical procedures by way of x-rays may be carried out as fluoroscopy, which is why the x-ray images may also be referred to as fluoroscopy images. In the case of real-time guidance, three-dimensional image data records (CT, magnetic resonance imaging, etc.) recorded preoperatively may be superposed onto the two-dimensional fluoroscopic x-ray images, with the accuracy of the superposition being critical for the clinical applicability.
To this end, the prior art has disclosed the practice of carrying out a 2D/3D registration at the start of the procedure or the fluoroscopic monitoring in order to provide the original accuracy. However, the 2D/3D registration may become invalid during the medical procedure due to movements of the patient, e.g., of the target region; consequently, incorrect superpositions may occur. In order to remove this problem, it is common practice for the person carrying out the procedure to be able to start a new registration procedure manually if the incorrect superposition becomes visible and it is consequently already possible to influence the interventional procedure. Then, the most recently recorded x-ray image is used to once again bring about a 2D/3D registration. A disadvantage here is that the person carrying out the procedure is interrupted when carrying out the interventional procedure by the movement correction.
A further approach for keeping the 2D/3D registration current provides for the patient or at least the target region to be tracked automatically during x-ray monitoring; consequently, there may be a movement correction “on-the-fly”. If the movement development is considered over time, it is also possible to predict the movement for future x-ray images. However, these tracking-based approaches also require a valid, accurate 2D/3D registration at the outset in order to obtain correspondences for estimating the three-dimensional movement. However, the error occurring in the movement estimation is disadvantageously identifiable in the fusion or superposition images after the tracking errors have accumulated and 2D/3D correspondences are lost by the movement.
Different central factors of the 2D/3D registration may be identified. Initially, the employed similarity measure (e.g., comparison measure) that describes the accuracy of the 2D/3D registration may be mentioned as a first important factor here. Use may be made of the image intensities (e.g., grayscale values) and gradients. The predominantly employed method for 2D/3D registration in accordance with the prior art is the establishment of a digitally reconstructed radiograph (DRR), with this being a simulated x-ray image for the three-dimensional image data record, which is consequently established by a forward projection; in this respect, (see, e.g., A. Kubias et al., “Extended global optimization strategy for rigid 2D/3D image registration”, CAIP, LINCS, volume 4673, Springer (2007), pages 759-767). Comparison measures based on the intensity values of the DRR image and of the x-ray image may be used as a similarity measure, for example, the sum of absolute differences (SAD), the sum of squared differences (SSD), and the normalized cross correlation (NCC). Examples for gradient-based comparison measures are the gradient cross correlation (see W. Wein et al., “2D/3D registration based on volume gradients”, Medical Imaging SPIE (2005), pages 144-150) and the normalized gradient field (NGF) (see E. Haber and J. Modersitzki, “Intensity gradient based registration and fusion of multi-modal images”, MICCAI 2006). These measures may be considered to be a more robust alternative to the intensity-based comparison measures.
Furthermore, the formulated movement model is relevant, e.g., in view of movements occurring in the target region. In general, the image registration is known as an optimization method, in which movement is applied to the moving image in order to bring it into correspondence with the stationary image. Here, the movement may be formulated for rigid structures as a rigid movement and an articulated (e.g., partly rigid) movement, but also as an elastic movement in which deformations may consequently also occur. From the point of view of the dimensionality, three-dimensional movements are applied to the three-dimensional image data record or two-dimensional movements are applied to the (e.g., projected) two-dimensional x-ray image (e.g., DRR). For examples in this respect, reference is made to the review article R. Liao et al., “A review of recent advances in registration techniques applied to minimally invasive therapy”, IEEE Transactions on Multimedia, 15(5), 2013, pages 983-1000.
The optimization strategy may be mentioned as a third important factor. A numerical optimization method is carried out on the basis of the selected comparison measure (e.g., similarity measure) and the movement model, the optimization measure using movement parameters of the movement model as parameters to be optimized in order to maximize the similarity or in order to minimize the difference. Here, use may be made of gradient-based optimization methods in respect of the comparison measure. However, it is also possible to use gradient-free optimization methods. By way of example, A. Kubias et al., in the already cited article “Extended global optimization strategy for rigid 2D/3D image registration”, propose to use an extended global optimization strategy, where an adaptive random search is used in various approximate resolution levels and a local optimizer is applied to a higher resolution level in order to refine the registration. In the tracking-based approaches, the optimization is often based on mathematical relationships between the tracked features and the three-dimensional rigid movement model.
Despite all these approaches, the 2D/3D registration still continues to pose a challenge, in particular when it is intended to be carried out in real time. A category of known methods attempts to enable the three-dimensional movement by comparing the projection of the three-dimensional image data record with the two-dimensional x-ray image; in this respect, see the aforementioned publications by A. Kubias et al., W. Wein et al., and E. Haber et al. However, the optimization is made more difficult in this case by the loss of the depth information occurring as a result of the projection. Another category of known methods uses the back projection of the x-ray image and compares the latter with the three-dimensional image data record, with these approaches, however, requiring a plurality of two-dimensional x-ray images recorded under different projection directions, e.g., in different recording geometries, which is often not the case in medical procedures.
The tracking-based approaches have the advantage of establishing the three-dimensional movement by two-dimensional tracking. However, they require a high-quality initialization and a re-initialization in order to originally establish or reestablish 2D/3D correspondences. DE 10 2013 214 479 A1 proposes a method for tracking a 2D/3D registration in the case of movement by virtue of contours being tracked in successively recorded projection images. For a contour point given in the two-dimensional x-ray image and an associated initial point in the three-dimensional image data record, it is possible to establish a target plane in which the contour point displaced due to the movement, the initial point displaced due to the movement and the focus, e.g., ultimately, the radiation source in the recording geometry, are situated. It is possible to establish a relationship that connects the observable degrees of freedom of movement perpendicular to the course of the contour in the three-dimensional image data record with three-dimensional descriptions of the movement, e.g., movement parameters of a movement model. This system of equations may be underdetermined for a single contour point. However, since a multiplicity of contour points are considered here, the result of this is a system of equations that may be solved in order to determine the movement parameters and in order consequently to update the movement information. What is exploited here is that the contour may be assigned to a rigid object that may consequently be mapped by common movement parameters describing a rigid movement. However, the problems of accumulating tracking errors and also the possible loss of 2D/3D correspondences also occur in this case.
A gradient-based method for rigid 2D/3D registration is proposed in an article by H. Livyatan et al. (“Gradient-based 2D/3D rigid registration of fluoroscopic X-ray to CT”, IEEE Transactions on Medical Imaging 22(11) 2003, pages 1395-1406). Here, an original pose is initially estimated; this is followed by carrying out an approximate geometry-based registration on bone contours and finally applying a gradient projection-based fine registration to edge pixels.